🛠 Exercises

1. Use the same evaluation techniques on the large-scale Food Vision model as you did in the previous notebook (Transfer Learning Part 3: Scaling up). More specifically, it would be good to see:

• A confusion matrix between all of the model's predictions and true labels.
• A graph showing the f1-scores of each class.
• A visualization of the model making predictions on various images and comparing the predictions to the ground truth.
  • For example, plot a sample image from the test dataset and have the title of the plot show the prediction, the prediction probability and the ground truth label.

2. Take 3 of your own photos of food and use the Food Vision model to make predictions on them. How does it go? Share your images/predictions with the other students.

3. Retrain the model (feature extraction and fine-tuning) we trained in this notebook, except this time use EfficientNetB4 as the base model instead of EfficientNetB0. Do you notice an improvement in performance? Does it take longer to train? Are there any tradeoffs to consider?

4. Name one important benefit of mixed precision training, how does this benefit take place?

1. Use evaluation technique on the food vision model

First bullet point states to compare confusion matrix between all different model's predictions - I will only do on our fine-tuned trained one as I don't see the point in recreating almost identical matrixes, where identifying the difference between 101 to 101 classes is unfeasible to conclude anything interesting, in addition to my limitation in hardware.

I will save a copy of the prediction on file from google colab, so it can be imported here to do comparisons on f1 score, and visualizing model prediction with image data.

Let's start by important some important tools to get our confusion matrix.

from helper_functions import make_confusion_matrix
import itertools
import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix

We will first import the csv file of the predictions, and check the head of it the data prediction as well.

# import csv and check head
import pandas as pd

df_preds = pd.read_csv('predictions.csv')
df_preds.head()

	prediction
0	29
1	81
2	91
3	53
4	97

Let's also import tensorflow data as well, and extract out the prediction data as well.

# gain access to food 101 predictions
import tensorflow_datasets as tfds

data_dir = r"X:\tensorflow_datasets"  # location of our food101 data from prev notebook

ds, ds_info = tfds.load(
    name='food101',
    split='validation',
    data_dir=data_dir,    
    shuffle_files=True,
    as_supervised=True,
    with_info=True
)

# extract y_true from the dataset
y_true=[]

for _, label in tfds.as_numpy(ds):
    y_true.append(label)

y_true[:5]

[83, 16, 60, 32, 36]

Mismatch of y_true and df_preds:

Because I pulled df_preds from google colab, and y_true formed from my pc, the problem we have here is suffle_files - this means I have to wait for google colab to fucking retrain everything yet again.

Note: I technically don't need to retrain the model because it was saved. But I'm concurrently training B4 however many times now for these dumbass exercises, and I'm not about to reset all B4 training progress to get this one dipshit fucking file. So I will use B4's predictions in place of B0, since I'm not about to wait another eternity on resetting everything again.

I've finally got the csv data of y_preds and y_true in one, and can now import it.

# import tge y pred and y true, now it's properly aligned
y_test_df = pd.read_csv('predictions.csv')
y_test_df.head()

	y_true	y_pred
0	83	83
1	16	16
2	60	60
3	32	32
4	36	36

Yup, seems like it's properly aligned! Let's form it into a confusion matrix.

But also, we'll need to get the class names of our food 101

# get class_name variable
class_names = ds_info.features['label'].names
class_names[:10]

['apple_pie',
 'baby_back_ribs',
 'baklava',
 'beef_carpaccio',
 'beef_tartare',
 'beet_salad',
 'beignets',
 'bibimbap',
 'bread_pudding',
 'breakfast_burrito']

# create confusion matrix

make_confusion_matrix(y_true=y_test_df['y_true'],
                      y_pred=y_test_df['y_pred'],
                      classes=class_names,
                      figsize=(100,100),
                      text_size=20,
                      norm=False, # no normalizing of values between 0 and 1
                      # this way, we can see the highest performing class, versus lowest
                      savefig=True) # save figure as image

Great, we got the confusion matrix. But it's completely useless, because there is no quantifiable way to create a conclusion with so many data points. And sure we know the model has done quite well overall on unseen data. But we don't know what were the best or worst performing food classes. And because of how high the colour values are for the correctly predicted classes, the small wrong prediction isolated spots don't have enough instances to really get recognized by the colour scale, making it hard to determine how many time Class A gets confused with Class B.

Solution: What I'd plan to do is plot a bar graph for class accuracy, and a confusion matrix of the 10 worst performing classes to get a better idea of model performance.

Our first step is to get our accuracy values of our prediction and y test data, which will be done by extracting our f1-score, which combines recall and precision scores together. The higher the score, the more accurate.

# get our accuracy value of each class via f1-score
from sklearn.metrics import classification_report

accuracy_report_dict = classification_report(y_test_df['y_pred'],y_test_df['y_true'], output_dict=True)

# create a dictionary to store f1_score of each class
class_f1_scores={}

for key, fscore in accuracy_report_dict.items():
    if key == 'accuracy': # stop the loop if we reach to 'accuracy' row
        break
    else: 
        # append class names and f1_score into new dict
        class_f1_scores[class_names[int(key)]] = fscore['f1-score'] # get classname via `key`, and get value from `f1-score`
class_f1_scores

{'apple_pie': 0.6943866943866944,
 'baby_back_ribs': 0.8352941176470589,
 'baklava': 0.8853754940711462,
 'beef_carpaccio': 0.8714859437751004,
 'beef_tartare': 0.7864271457085829,
 'beet_salad': 0.7443762781186094,
 'beignets': 0.8832684824902723,
 'bibimbap': 0.9224652087475149,
 'bread_pudding': 0.680327868852459,
 'breakfast_burrito': 0.8126232741617357,
 'bruschetta': 0.782435129740519,
 'caesar_salad': 0.8837209302325582,
 'cannoli': 0.8952772073921971,
 'caprese_salad': 0.824,
 'carrot_cake': 0.832,
 'ceviche': 0.718816067653277,
 'cheesecake': 0.74,
 'cheese_plate': 0.8884381338742393,
 'chicken_curry': 0.7717842323651453,
 'chicken_quesadilla': 0.8600823045267489,
 'chicken_wings': 0.8858267716535433,
 'chocolate_cake': 0.7327935222672065,
 'chocolate_mousse': 0.6567164179104478,
 'churros': 0.9087301587301587,
 'clam_chowder': 0.9094567404426559,
 'club_sandwich': 0.8980392156862745,
 'crab_cakes': 0.7427466150870407,
 'creme_brulee': 0.9111969111969112,
 'croque_madame': 0.8995983935742972,
 'cup_cakes': 0.9007936507936508,
 'deviled_eggs': 0.9212598425196851,
 'donuts': 0.8594059405940594,
 'dumplings': 0.900398406374502,
 'edamame': 0.9880952380952381,
 'eggs_benedict': 0.878727634194831,
 'escargots': 0.8957055214723927,
 'falafel': 0.8104838709677419,
 'filet_mignon': 0.679920477137177,
 'fish_and_chips': 0.8927875243664717,
 'foie_gras': 0.6666666666666666,
 'french_fries': 0.916504854368932,
 'french_onion_soup': 0.8638132295719845,
 'french_toast': 0.7868852459016393,
 'fried_calamari': 0.8611111111111112,
 'fried_rice': 0.8678500986193294,
 'frozen_yogurt': 0.931237721021611,
 'garlic_bread': 0.8016032064128257,
 'gnocchi': 0.78125,
 'greek_salad': 0.8532289628180039,
 'grilled_cheese_sandwich': 0.7992202729044834,
 'grilled_salmon': 0.7881188118811882,
 'guacamole': 0.924901185770751,
 'gyoza': 0.889344262295082,
 'hamburger': 0.849802371541502,
 'hot_and_sour_soup': 0.9428007889546351,
 'hot_dog': 0.9149797570850202,
 'huevos_rancheros': 0.7383966244725738,
 'hummus': 0.8194174757281554,
 'ice_cream': 0.7897838899803536,
 'lasagna': 0.7857142857142857,
 'lobster_bisque': 0.8705882352941177,
 'lobster_roll_sandwich': 0.9236947791164659,
 'macaroni_and_cheese': 0.8512396694214877,
 'macarons': 0.9540918163672655,
 'miso_soup': 0.9336016096579477,
 'mussels': 0.9252525252525252,
 'nachos': 0.8473895582329317,
 'omelette': 0.8024439918533605,
 'onion_rings': 0.908,
 'oysters': 0.9482071713147411,
 'pad_thai': 0.9213051823416507,
 'paella': 0.8313253012048193,
 'pancakes': 0.8779527559055118,
 'panna_cotta': 0.8259109311740891,
 'peking_duck': 0.8819875776397516,
 'pho': 0.9263565891472868,
 'pizza': 0.9168278529980658,
 'pork_chop': 0.6749482401656315,
 'poutine': 0.8924949290060852,
 'prime_rib': 0.8526522593320236,
 'pulled_pork_sandwich': 0.8543307086614174,
 'ramen': 0.8993839835728953,
 'ravioli': 0.7379454926624738,
 'red_velvet_cake': 0.8923679060665362,
 'risotto': 0.7800829875518672,
 'samosa': 0.8722109533468559,
 'sashimi': 0.91796875,
 'scallops': 0.7689161554192229,
 'seaweed_salad': 0.932,
 'shrimp_and_grits': 0.8095238095238095,
 'spaghetti_bolognese': 0.9301397205588823,
 'spaghetti_carbonara': 0.9580838323353293,
 'spring_rolls': 0.884,
 'steak': 0.5413223140495868,
 'strawberry_shortcake': 0.8466019417475729,
 'sushi': 0.8582995951417004,
 'tacos': 0.7647058823529411,
 'takoyaki': 0.9243027888446215,
 'tiramisu': 0.7943548387096774,
 'tuna_tartare': 0.7087576374745418,
 'waffles': 0.900398406374502}

Now we can form the data into a pandas dataframe, and organize them descending order!

# create pandas df and put then in descending order
import pandas as pd

f1_scores = pd.DataFrame({'class_names': list(class_f1_scores.keys()), # list all keys aka our class names 
                        'f1-score': list(class_f1_scores.values())}).sort_values('f1-score', ascending=False) # list all values, and make sure ascention is false
f1_scores.head()

	class_names	f1-score
33	edamame	0.988095
91	spaghetti_carbonara	0.958084
63	macarons	0.954092
69	oysters	0.948207
54	hot_and_sour_soup	0.942801

Now time to create a horizontal bar chart with all the accuracy sorted in descending order.

# create horizontal bar graph of accuracy results
fig, ax = plt.subplots(figsize=(12,25))
scores = ax.barh(range(len(f1_scores)), f1_scores['f1-score'].values) # get accuracy score values for bar heights
ax.set_yticks(range(len(f1_scores))) # make sure y axis is same length as number of unique classes
ax.set_yticklabels(list(f1_scores['class_names'])) # get class name for y label
ax.set_xlabel('f1-score')
ax.set_title('F1 scores for all 101 classes in descending order')
ax.invert_yaxis(); # reverse bar graph order

def autolabel(rects):
    '''
    Attach label above each bar, displaying accuracy
    '''
    for rect in rects:
        width=rect.get_width() # get size of bar
        ax.text(1.03*width, rect.get_y() + rect.get_height()/1.5, # positioning of text so it's readable and aligned
                f'{width:.2f}', # 2 significant figures for the accuracy
                ha='center')

autolabel(scores)

For pretty much every class except one, the model does better than just 50/50 random guessing. We can see how the model has struggle with some specific food categories, compared to others. Which tells us that we may need to gather more data and do some specific focusing on the worst performing categories, especially steak.

With such bad performance, my first thought into why is Food A may confuse the image with Food B. Because of how varied food dishes are, and it's presentation, inevitably some may look very similar to another. Aka pies like fruit apple pie, and meat cheese pies.

So what I'll do is make a confusion matrix. But instead of all classes, only display the 10 worst predicting ones, so we get a better visual idea of what the class is getting confused with (while obscuring the true predictions, so the false predictions will stand out more in colour).

import matplotlib.pyplot as plt
import numpy as np
from sklearn.metrics import confusion_matrix

def focused_confusion_matrix_top10_correct(y_true, y_pred, f1_scores, class_names, n_worst=10, figsize=(18, 6), cmap='Reds'):
    """
    Confusion matrix for the n_worst classes with lowest F1-score.
    True axis: only worst classes
    Pred axis: all classes
    """
    # Step 1: get indices of the n_worst lowest F1-scores
    index = f1_scores.index.tolist()
    worst_indices = index[-10:]

    # Step 2: compute confusion matrix for all classes
    cm_full = confusion_matrix(y_true, y_pred, labels=np.arange(len(class_names)))

    # Step 3: extract only worst rows
    cm_focus = cm_full[worst_indices, :].copy()

    # Step 4: zero out diagonal for these rows
    for i, idx in enumerate(worst_indices):
        cm_focus[i, idx] = 0

    # Step 5: plot
    plt.figure(figsize=figsize)
    im = plt.imshow(cm_focus, interpolation='nearest', cmap=cmap)
    plt.colorbar(im, label='Misclassifications', orientation='horizontal', pad=0.2)
    plt.title(f"Confusion Matrix: Worst {n_worst} Classes (Ordered by F1)")

    # X-axis: all class names
    plt.xticks(np.arange(len(class_names)), class_names, rotation=90, fontsize=6)
    # Y-axis: only worst classes, ordered worst → better
    plt.yticks(np.arange(n_worst), [class_names[i] for i in worst_indices], fontsize=8)

    # Add numbers
    for i in range(n_worst):
        for j in range(len(class_names)):
            if cm_focus[i, j] > 0:
                plt.text(j, i, str(cm_focus[i, j]),
                         ha="center", va="center", color="black", fontsize=4)

    plt.ylabel('True Class (Worst)')
    plt.xlabel('Predicted Class')
    plt.tight_layout()
    plt.show()

Above code is a few dozen generations of code via ChatGPT, to speed up the process and get straight into the data. Tried to sort it by worst performing instead of alphabetical order, gave the fuck up and just settle with what I have.

I'VE BEEN FUCKING SHITTING ON THIS SHIT FOR 2+ FUCKING HOURS TO GET IT ORGANIZED BY LOWEST FUCKING F1-SCORE WHEN IN ACTUALITY IT'S SHOWING ME THE LAST 10 RESULTS, FUCK MY LIFE THIS TOOK FUCKING FOREVER, FUCK CHATGPT

Next day: After getting some much needed rest, I'm going to fix up the selection of the 10 classes. I used a simple list of the known index locality of all 10 worst performing class just to get the job done:

worst_indices = [93, 22, 39, 77, 37, 8, 0, 99, 15, 21]

The code to fix this simple shitty issue is as simple as:

index = f1_scores.index.tolist() # get index variables from our f1-score list, that's already organized in descending order
    worst_indices = index[-10:] # get the last 10 results

Thank god that's done.

# Then call the function
focused_confusion_matrix_top10_correct(
    y_true=y_test_df['y_true'].to_numpy(),
    y_pred=y_test_df['y_pred'].to_numpy(),
    f1_scores=f1_scores,
    class_names=class_names,    # list of 101 class names in order
    n_worst=10,
    figsize=(20, 8)
)

Starting from the worst predicting food, steak, right off the bat we see one that stands out amongst all of them. Filet mignon. Let's get a random ass picture and compare steak to filet mignon side by side.

import matplotlib.pyplot as plt
import matplotlib.image as mpimg

img1 = mpimg.imread("garlic-butter-filet-mignon-973408.jpg")
img2 = mpimg.imread("Steak_and_rocket_on_plate.webp")

fig, axes = plt.subplots(1, 2, figsize=(12, 15))

axes[0].imshow(img1)
axes[1].imshow(img2)

for ax in axes:
    ax.axis("off")

plt.tight_layout()
plt.show()

Honestly they look exactly the same - both made from cow anyways. Just one looks a little more rounder and circular than the other. Left is Filet Mignon, Right is Steak.

So easily based on that, you can't really blame the model for getting it wrong. Because of that, most likely filet mignon also has some wrong predictions of being identified as steak as well.(we can actually see the next reddest spot is indeed confusion for steak instead of filet mignon)

We can also see the other highlighted ones are baby_back_rib, pork_chop, and prime_rib. All with very similar presentation and cooking profiles, with similar cut of meat that may even come from cows as well.

There are probably 3 more food items that get confused with each other. Bread_pudding to Apple_pie, Chocolate_mousse to Chcolate_cake, and Beef_tatare to Tuna_tatare.

And here are the following foods by that specified order:

img1 = mpimg.imread("cinnamon-roll-bread-pudding-featured2.webp")
img2 = mpimg.imread("oynyDlfN.jpg")
img3 = mpimg.imread("chocolate-mousse-featured.webp")
img4 = mpimg.imread("8Mhboqg7.jpg")
img5 = mpimg.imread("Classic_steak_tartare.jpg")
img6 = mpimg.imread("images-1.jpg")

fig, axes = plt.subplots(2, 4, figsize=(20, 10))

axes[0,0].imshow(img1)
axes[0,1].imshow(img2)
axes[0,2].imshow(img3)
axes[0,3].imshow(img4)
axes[1,1].imshow(img5)
axes[1,2].imshow(img6)

for ax in axes.flat:
    ax.axis("off")

plt.tight_layout()
plt.show()

It's also quite fitting, how the most confused foods for another food, all sit within the bottom 10 accuracy as well, showing how difficult it is to differentiate these categories of food. This tells us that we could benefit in making a model, that aims to differentiate a Filet mignon to a Steak, or Bread pudding to an Apple pie.

Next, I want to check out not just f1-score, but the following recall and precision scores of the 10 worst performing classes via their f1-scores.

import pandas as pd

accuracy_report_dict = classification_report(y_test_df['y_pred'],y_test_df['y_true'], output_dict=True)

# Extract class-level metrics into a DataFrame
metrics_list = []
for key, scores in accuracy_report_dict.items():
    if key == 'accuracy':
        break  # skip overall accuracy
    metrics_list.append({
        'class_name': class_names[int(key)],   # map key to class name
        'precision': scores['precision'],
        'recall': scores['recall'],
        'f1_score': scores['f1-score']
    })

metrics_df = pd.DataFrame(metrics_list)

# Get 10 worst classes by F1-score
worst_10_df = metrics_df.nsmallest(10, 'f1_score')

# Display
print(worst_10_df)

          class_name  precision    recall  f1_score
93             steak      0.524  0.559829  0.541322
22  chocolate_mousse      0.616  0.703196  0.656716
39         foie_gras      0.648  0.686441  0.666667
77         pork_chop      0.652  0.699571  0.674948
37      filet_mignon      0.684  0.675889  0.679920
8      bread_pudding      0.664  0.697479  0.680328
0          apple_pie      0.668  0.722944  0.694387
99      tuna_tartare      0.696  0.721992  0.708758
15           ceviche      0.680  0.762332  0.718816
21    chocolate_cake      0.724  0.741803  0.732794

From the looks of it, both precision and recall have similar standings for every 10 worst performing class. It does appear that recall (where we calculate all true positive divided by all the actual positives [TP + FN]), tend to be slightly higher than our precision (where we calculate all true positive divided by all the positives the model has predicted [TP + FP]). The only exception to that rule is Filet mignon, with a slightly higher precision than recall.

This does mean our model is slightly better at it's job at detecting all possible images that are labeled as it's original class, with the slight sacrifice of accidentally predicting too much, creating False Positives in our data.

This brings me to the next question, how is the model's distribution in prediction look like? I want to plot a bar graph, showing how many times the model has predicted each class.

import matplotlib.pyplot as plt

# Make sure worst_classes are class names
# Extract class names as a list
class_names = f1_scores['class_names'].tolist()
worst_class_names = class_names[-10:]
best_class_names = class_names[:10]

# Make sure both are strings and stripped
pred_class_names = [str(cls).strip() for cls in pred_counts.index]
worst_class_names_clean = [str(cls).strip() for cls in worst_class_names]

colors = ['red' if cls in worst_class_names_clean else 'green' if cls in best_class_names else 'blue' for cls in pred_class_names]

plt.figure(figsize=(18,6))
plt.bar(pred_class_names, pred_counts.values, color=colors)
plt.xticks(rotation=90)
plt.ylabel("Number of times predicted")
plt.title("Number of Predictions per Class (Worst Classes Highlighted)")
plt.show()

Here, the labeled colour bars mean it is either the 10 worst predicting classes (red), or the 10 best predicting classes (green). I would've thought red would be at the lower end, whilst green at the higher end. But it appears to have a lack of pattern and is quite normally distributed.

The model does a pretty good job at predicting each class at around 250, which is the number of image samples per class. It does have a very slight tendency to predict a little more for the majority of the classes, and a few is left trailing at the end.

My thoughts is that the lower end, tend to have a very unique appearance that most foods don't have, making it difficult to identify. Vice versa for the food on the other end.

Here is clam_and_chowder, pad_thai, and greek_salad:

img1 = mpimg.imread("13078-New-England-Clam-Chowder-DDMFS-3X4-0560-e9b8d6db885546538990735f2d4d2199.jpg")
img2 = mpimg.imread("Untitled-design-5.jpg")
img3 = mpimg.imread('images-2.jpg')

fig, axes = plt.subplots(1, 3, figsize=(12, 15))

axes[0].imshow(img1)
axes[1].imshow(img2)
axes[2].imshow(img3)

for ax in axes:
    ax.axis("off")

plt.tight_layout()
plt.show()

Then here's shrimp and grits, waffles, and dumplings:

img1 = mpimg.imread("shrimp-and-grits-hero.webp")
img2 = mpimg.imread("20513-classic-waffles-mfs-025-4x3-81c0f0ace44d480ca69dd5f2c949731a.jpg")
img3 = mpimg.imread('images-3.jpg')

fig, axes = plt.subplots(1, 3, figsize=(12, 15))

axes[0].imshow(img1)
axes[1].imshow(img2)
axes[2].imshow(img3)

for ax in axes:
    ax.axis("off")

plt.tight_layout()
plt.show()

They do slightly follow that pattern of uniqueness vs. ordinary/simple

So our next step is to visualize the actual predictions the model has made. It helps to know what type of food classes the model is struggling with, but better if we can visualize it's actual performance.

What we'll do is get the top 16 most confident wrong predictions, and see if there are patterns outside of what we've found.

We will first get the filepath names of all test images.

Visualizing worst predictions is impossible

Because how our predictions data came straight from google colab, we only had the predictions column, along with the y label column. We don't have the predictions probability of our top class, therefore we have no way in truly identifying the most confident predictions of the wrong classes.

Not only that, but we've had shuffle_files=True, which shuffled the tfrecords. What it is, is more like gigantic preset batches of say 0-999 images, then the next is 1000-1999 images. This changed alphabetical ordering of the images, like steak before apple pie.

The biggest factor is parallel preprocessing. Because of the constant input of reading batches in between predicting batches, naturally some batches will finish prediction than others, leading to a very mixed up order of test data. The input of test data may be organized, but the output is completely disordered because of the way parallel preprocessing does things. This means we will always get a shuffled predictions when running our evaluataion no matter what.

So we neither have prediction probabilities, or properly set index orders of the images, making it impossible to truly identify the strongest predictions by the model, that turns out to be wrong.

2. Import 3 custom photos, and get the model to predict on them

For the sake of pure laziness, time saving, but also for experimentation consistency, I will utilize the 3 custom photos I've used for 06 notebook exercise, and do it for the exercise here.

Our list of steps:

• Get custom image loc
• load our model
• get class_names
• preprocess image function
• predict images and visually display them

Let's see whether a more accurate ML model can make a more reasonable prediction than the previous one.

# get filepath of custom image
import os

custom_food = ['../06 notebook/my_custom_food_image/' + img_path for img_path in os.listdir('../06 notebook/my_custom_food_image/')]
custom_food

['../06 notebook/my_custom_food_image/20251113_222110.jpg',
 '../06 notebook/my_custom_food_image/20251113_222254.jpg',
 '../06 notebook/my_custom_food_image/20251113_222134.jpg']

Let's predict them with the model, by loading it in, and plotting them out

# load in model
import tensorflow as tf
model = tf.keras.models.load_model('downloaded_model/07_efficientnetb0_fine_tuned_101_classes_mixed_precision')

WARNING:tensorflow:SavedModel saved prior to TF 2.5 detected when loading Keras model. Please ensure that you are saving the model with model.save() or tf.keras.models.save_model(), *NOT* tf.saved_model.save(). To confirm, there should be a file named "keras_metadata.pb" in the SavedModel directory.
WARNING:tensorflow:Mixed precision compatibility check (mixed_float16): WARNING
The dtype policy mixed_float16 may run slowly because this machine does not have a GPU. Only Nvidia GPUs with compute capability of at least 7.0 run quickly with mixed_float16.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once
WARNING:absl:Importing a function (__inference_block3b_expand_activation_layer_call_and_return_conditional_losses_443625) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5c_activation_layer_call_and_return_conditional_losses_412189) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block1a_se_reduce_layer_call_and_return_conditional_losses_409120) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6c_expand_activation_layer_call_and_return_conditional_losses_446895) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block1a_activation_layer_call_and_return_conditional_losses_442329) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_efficientnetb0_layer_call_and_return_conditional_losses_421687) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5b_activation_layer_call_and_return_conditional_losses_411851) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6b_se_reduce_layer_call_and_return_conditional_losses_446637) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6d_activation_layer_call_and_return_conditional_losses_447353) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5b_expand_activation_layer_call_and_return_conditional_losses_445426) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block1a_se_reduce_layer_call_and_return_conditional_losses_442371) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_efficientnetb0_layer_call_and_return_conditional_losses_419945) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2a_se_reduce_layer_call_and_return_conditional_losses_442703) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5a_activation_layer_call_and_return_conditional_losses_445173) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_efficientnetb0_layer_call_and_return_conditional_losses_440876) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block7a_se_reduce_layer_call_and_return_conditional_losses_447774) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block7a_expand_activation_layer_call_and_return_conditional_losses_413771) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6c_se_reduce_layer_call_and_return_conditional_losses_413200) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4b_expand_activation_layer_call_and_return_conditional_losses_410834) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4c_se_reduce_layer_call_and_return_conditional_losses_411277) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3b_expand_activation_layer_call_and_return_conditional_losses_410210) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5a_se_reduce_layer_call_and_return_conditional_losses_445215) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6c_activation_layer_call_and_return_conditional_losses_446974) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3a_se_reduce_layer_call_and_return_conditional_losses_443414) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6d_expand_activation_layer_call_and_return_conditional_losses_413433) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_stem_activation_layer_call_and_return_conditional_losses_442250) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4b_activation_layer_call_and_return_conditional_losses_410890) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block7a_se_reduce_layer_call_and_return_conditional_losses_413876) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_top_activation_layer_call_and_return_conditional_losses_414056) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3a_activation_layer_call_and_return_conditional_losses_409981) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_efficientnetb0_layer_call_and_return_conditional_losses_435905) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3b_activation_layer_call_and_return_conditional_losses_410266) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3a_expand_activation_layer_call_and_return_conditional_losses_409924) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4b_se_reduce_layer_call_and_return_conditional_losses_410939) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4c_activation_layer_call_and_return_conditional_losses_444794) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6b_se_reduce_layer_call_and_return_conditional_losses_412862) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5c_se_reduce_layer_call_and_return_conditional_losses_445926) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference__wrapped_model_403250) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5b_se_reduce_layer_call_and_return_conditional_losses_445547) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6d_expand_activation_layer_call_and_return_conditional_losses_447274) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6a_expand_activation_layer_call_and_return_conditional_losses_412471) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5c_se_reduce_layer_call_and_return_conditional_losses_412238) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5a_expand_activation_layer_call_and_return_conditional_losses_411510) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5c_activation_layer_call_and_return_conditional_losses_445884) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5a_se_reduce_layer_call_and_return_conditional_losses_411615) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3b_se_reduce_layer_call_and_return_conditional_losses_410315) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4b_expand_activation_layer_call_and_return_conditional_losses_444336) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_model_layer_call_and_return_conditional_losses_429172) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2a_se_reduce_layer_call_and_return_conditional_losses_409406) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6a_se_reduce_layer_call_and_return_conditional_losses_412577) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_efficientnetb0_layer_call_and_return_conditional_losses_434163) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_top_activation_layer_call_and_return_conditional_losses_447985) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6b_expand_activation_layer_call_and_return_conditional_losses_412757) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6a_expand_activation_layer_call_and_return_conditional_losses_446184) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5a_activation_layer_call_and_return_conditional_losses_411566) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4b_activation_layer_call_and_return_conditional_losses_444415) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2b_expand_activation_layer_call_and_return_conditional_losses_442914) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2a_expand_activation_layer_call_and_return_conditional_losses_409300) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3a_activation_layer_call_and_return_conditional_losses_443372) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6d_se_reduce_layer_call_and_return_conditional_losses_447395) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5a_expand_activation_layer_call_and_return_conditional_losses_445094) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6d_activation_layer_call_and_return_conditional_losses_413489) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4a_expand_activation_layer_call_and_return_conditional_losses_410548) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6b_activation_layer_call_and_return_conditional_losses_412813) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4a_activation_layer_call_and_return_conditional_losses_444083) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4c_se_reduce_layer_call_and_return_conditional_losses_444836) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block7a_activation_layer_call_and_return_conditional_losses_447732) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5b_expand_activation_layer_call_and_return_conditional_losses_411795) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6b_expand_activation_layer_call_and_return_conditional_losses_446516) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4a_se_reduce_layer_call_and_return_conditional_losses_444125) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4c_expand_activation_layer_call_and_return_conditional_losses_444715) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2b_se_reduce_layer_call_and_return_conditional_losses_443035) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2a_activation_layer_call_and_return_conditional_losses_442661) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_model_layer_call_and_return_conditional_losses_430926) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4a_expand_activation_layer_call_and_return_conditional_losses_444004) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6c_expand_activation_layer_call_and_return_conditional_losses_413095) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5b_se_reduce_layer_call_and_return_conditional_losses_411900) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6c_se_reduce_layer_call_and_return_conditional_losses_447016) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6a_activation_layer_call_and_return_conditional_losses_412528) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3b_se_reduce_layer_call_and_return_conditional_losses_443746) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2b_activation_layer_call_and_return_conditional_losses_442993) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3a_expand_activation_layer_call_and_return_conditional_losses_443293) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6a_se_reduce_layer_call_and_return_conditional_losses_446305) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6b_activation_layer_call_and_return_conditional_losses_446595) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block7a_activation_layer_call_and_return_conditional_losses_413827) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3b_activation_layer_call_and_return_conditional_losses_443704) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2a_activation_layer_call_and_return_conditional_losses_409357) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6c_activation_layer_call_and_return_conditional_losses_413151) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block3a_se_reduce_layer_call_and_return_conditional_losses_410030) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4b_se_reduce_layer_call_and_return_conditional_losses_444457) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block1a_activation_layer_call_and_return_conditional_losses_409071) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5c_expand_activation_layer_call_and_return_conditional_losses_445805) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4c_activation_layer_call_and_return_conditional_losses_411228) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2a_expand_activation_layer_call_and_return_conditional_losses_442582) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4a_se_reduce_layer_call_and_return_conditional_losses_410654) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block7a_expand_activation_layer_call_and_return_conditional_losses_447653) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6a_activation_layer_call_and_return_conditional_losses_446263) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block6d_se_reduce_layer_call_and_return_conditional_losses_413538) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5c_expand_activation_layer_call_and_return_conditional_losses_412133) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2b_se_reduce_layer_call_and_return_conditional_losses_409691) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2b_expand_activation_layer_call_and_return_conditional_losses_409586) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block5b_activation_layer_call_and_return_conditional_losses_445505) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4a_activation_layer_call_and_return_conditional_losses_410605) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_stem_activation_layer_call_and_return_conditional_losses_409015) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block2b_activation_layer_call_and_return_conditional_losses_409642) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_block4c_expand_activation_layer_call_and_return_conditional_losses_411172) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_efficientnetb0_layer_call_and_return_conditional_losses_439134) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.

Before we make a prediction, let's also get class names so we know what is being predicted.

# get class_name variable
class_names = ds_info.features['label'].names
class_names[:10]

['apple_pie',
 'baby_back_ribs',
 'baklava',
 'beef_carpaccio',
 'beef_tartare',
 'beet_salad',
 'beignets',
 'bibimbap',
 'bread_pudding',
 'breakfast_burrito']

We will also have to preprocess the images, so that it is actually interpretable for the model.

def preprocess_img(path, img_shape=224):
    img=tf.io.read_file(path) # read our image
    img = tf.image.decode_jpeg(img, channels=3) # decode image to a tensor - readable format for model
    img=tf.image.resize(img, (img_shape,img_shape)) # resize img to 224 224
    img=tf.cast(img,tf.float32)/255.0 # also changing image uint8 datatype to float32 - keep pixel val between 0 and 1
    return img

# make prediction and output visually as well
for img in custom_food:
    img = preprocess_img(img)
    pred_prob = model.predict(tf.expand_dims(img,axis=0)) # expand dims for batch num placement
    pred_class = class_names[pred_prob.argmax()] # get only max prediction value

    # plot image
    plt.figure()
    plt.imshow(img)
    plt.title(f'Prediction: {pred_class}, Probability: {pred_prob.max():.2f}')
    plt.axis(False)

1/1 [==============================] - 0s 336ms/step
1/1 [==============================] - 0s 315ms/step
1/1 [==============================] - 0s 274ms/step

Oh wow... seems like predictions are much worse with this more accurate model lol.

There's definitely some bug with the Nan prediction, and applie_pie may not be the prediction, as it is the first indexed class name. Something has likely gone wrong however I have not been able to figure it out despite some fiddling.

But yea, the model made on this computer from workbook 06 did a much more reasonable prediction than whatever this model did.

3. Retrain model with EfficientNetB4 instead of EfficientNetB0

The exercise states us to train everything again including the fine tuning process - I thought I could assume to extrapolate info from feature extraction, onto fine tuning and skip having to wait for the training of both models. But a little consolation with ChatGPT, has begrudgingly changed my mind.

Here are the following results for the 3 epochs feature extraction >

B4 Feature Extraction

Saving TensorBoard log files to: training_logs/efficientnetb4_101_classes_all_data_feature_extract/20251203-054554
Epoch 1/3
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 375s 116ms/step - accuracy: 0.4543 - loss: 2.3221 - val_accuracy: 0.6674 - val_loss: 1.2579
Epoch 2/3
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 192s 80ms/step - accuracy: 0.6321 - loss: 1.4293 - val_accuracy: 0.6896 - val_loss: 1.1742
Epoch 3/3
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 201s 80ms/step - accuracy: 0.6694 - loss: 1.2679 - val_accuracy: 0.6994 - val_loss: 1.1048

B0 Feature Extraction

Saving TensorBoard log files to: training_logs/efficientnetb0_101_classes_all_data_feature_extract/20251202-032308
Epoch 1/3
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 279s 92ms/step - accuracy: 0.4771 - loss: 2.2929 - val_accuracy: 0.6976 - val_loss: 1.1339
Epoch 2/3
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 221s 69ms/step - accuracy: 0.6670 - loss: 1.2833 - val_accuracy: 0.7097 - val_loss: 1.0366
Epoch 3/3
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 169s 70ms/step - accuracy: 0.7032 - loss: 1.1371 - val_accuracy: 0.7246 - val_loss: 0.9935

B0 has done somewhat better than B4 in terms of of overall accuracy and loss performance. In addition to that, B4 has taken longer to train than B0, with total time taking B0 is 669 seconds, compared to B4's 768 seconds. Based on chatgpt's information, B4 is a larger model making training slightly longer, and prefers to use image size (380*380) instead of (224*224). Due to it's size, this does mean batch size may need to be reduced from 32 to 16 to accomodate memory usage.

Evaluation comparison 1

- B4
790/790 ━━━━━━━━━━━━━━━━━━━━ 104s 85ms/step - accuracy: 0.7020 - loss: 1.1066

- B0
790/790 ━━━━━━━━━━━━━━━━━━━━ 82s 75ms/step - accuracy: 0.7239 - loss: 1.0042

Based on our current evaluation, the conclusions made from feature extraction can still be applied here, with B0 slightly outperforming B4 in speed and accuracy.

B4 Fine tune

Saving TensorBoard log files to: training_logs/efficientb4_101_classes_all_data_fine_tuning/20251207-080846
Epoch 1/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 1043s 299ms/step - accuracy: 0.5897 - loss: 1.6187 - val_accuracy: 0.8130 - val_loss: 0.6699 - learning_rate: 1.0000e-04
Epoch 2/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 470s 197ms/step - accuracy: 0.8304 - loss: 0.6329 - val_accuracy: 0.8318 - val_loss: 0.6184 - learning_rate: 1.0000e-04
Epoch 3/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 461s 193ms/step - accuracy: 0.9097 - loss: 0.3211 - val_accuracy: 0.8294 - val_loss: 0.6335 - learning_rate: 1.0000e-04
Epoch 4/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 0s 191ms/step - accuracy: 0.9534 - loss: 0.1679

Epoch 4: ReduceLROnPlateau reducing learning rate to 1.9999999494757503e-05.

2368/2368 ━━━━━━━━━━━━━━━━━━━━ 463s 194ms/step - accuracy: 0.9534 - loss: 0.1679 - val_accuracy: 0.8310 - val_loss: 0.6675 - learning_rate: 1.0000e-04
Epoch 5/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 466s 196ms/step - accuracy: 0.9779 - loss: 0.0854 - val_accuracy: 0.8448 - val_loss: 0.6114 - learning_rate: 2.0000e-05
Epoch 6/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 459s 193ms/step - accuracy: 0.9892 - loss: 0.0502 - val_accuracy: 0.8488 - val_loss: 0.6172 - learning_rate: 2.0000e-05
Epoch 7/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 0s 188ms/step - accuracy: 0.9915 - loss: 0.0380

Epoch 7: ReduceLROnPlateau reducing learning rate to 3.999999898951501e-06.

2368/2368 ━━━━━━━━━━━━━━━━━━━━ 456s 191ms/step - accuracy: 0.9915 - loss: 0.0380 - val_accuracy: 0.8541 - val_loss: 0.6163 - learning_rate: 2.0000e-05
Epoch 8/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 457s 192ms/step - accuracy: 0.9944 - loss: 0.0296 - val_accuracy: 0.8559 - val_loss: 0.6125 - learning_rate: 4.0000e-06

B0 Fine Tune

Saving TensorBoard log files to: training_logs/efficientb0_101_classes_all_data_fine_tuning/20251209-071208
Epoch 1/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 566s 157ms/step - accuracy: 0.6238 - loss: 1.4720 - val_accuracy: 0.7987 - val_loss: 0.7210 - learning_rate: 1.0000e-04
Epoch 2/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 227s 95ms/step - accuracy: 0.7977 - loss: 0.7564 - val_accuracy: 0.8170 - val_loss: 0.6651 - learning_rate: 1.0000e-04
Epoch 3/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 261s 94ms/step - accuracy: 0.8545 - loss: 0.5362 - val_accuracy: 0.8242 - val_loss: 0.6453 - learning_rate: 1.0000e-04
Epoch 4/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 228s 95ms/step - accuracy: 0.9010 - loss: 0.3739 - val_accuracy: 0.8249 - val_loss: 0.6371 - learning_rate: 1.0000e-04
Epoch 5/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 226s 94ms/step - accuracy: 0.9292 - loss: 0.2686 - val_accuracy: 0.8361 - val_loss: 0.6288 - learning_rate: 1.0000e-04
Epoch 6/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 263s 94ms/step - accuracy: 0.9496 - loss: 0.2032 - val_accuracy: 0.8279 - val_loss: 0.6583 - learning_rate: 1.0000e-04
Epoch 7/100
2367/2368 ━━━━━━━━━━━━━━━━━━━━ 0s 90ms/step - accuracy: 0.9619 - loss: 0.1509

Epoch 7: ReduceLROnPlateau reducing learning rate to 1.9999999494757503e-05.

2368/2368 ━━━━━━━━━━━━━━━━━━━━ 223s 93ms/step - accuracy: 0.9619 - loss: 0.1509 - val_accuracy: 0.8294 - val_loss: 0.6628 - learning_rate: 1.0000e-04
Epoch 8/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 224s 94ms/step - accuracy: 0.9756 - loss: 0.1035 - val_accuracy: 0.8448 - val_loss: 0.6209 - learning_rate: 2.0000e-05
Epoch 9/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 224s 93ms/step - accuracy: 0.9820 - loss: 0.0821 - val_accuracy: 0.8408 - val_loss: 0.6296 - learning_rate: 2.0000e-05
Epoch 10/100
2367/2368 ━━━━━━━━━━━━━━━━━━━━ 0s 90ms/step - accuracy: 0.9849 - loss: 0.0716

Epoch 10: ReduceLROnPlateau reducing learning rate to 3.999999898951501e-06.

2368/2368 ━━━━━━━━━━━━━━━━━━━━ 225s 94ms/step - accuracy: 0.9849 - loss: 0.0716 - val_accuracy: 0.8438 - val_loss: 0.6233 - learning_rate: 2.0000e-05
Epoch 11/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 224s 94ms/step - accuracy: 0.9868 - loss: 0.0629 - val_accuracy: 0.8464 - val_loss: 0.6196 - learning_rate: 4.0000e-06
Epoch 12/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 222s 93ms/step - accuracy: 0.9894 - loss: 0.0571 - val_accuracy: 0.8435 - val_loss: 0.6286 - learning_rate: 4.0000e-06
Epoch 13/100
2367/2368 ━━━━━━━━━━━━━━━━━━━━ 0s 91ms/step - accuracy: 0.9900 - loss: 0.0556

Epoch 13: ReduceLROnPlateau reducing learning rate to 7.999999979801942e-07.

2368/2368 ━━━━━━━━━━━━━━━━━━━━ 265s 94ms/step - accuracy: 0.9900 - loss: 0.0556 - val_accuracy: 0.8416 - val_loss: 0.6372 - learning_rate: 4.0000e-06
Epoch 14/100
2368/2368 ━━━━━━━━━━━━━━━━━━━━ 221s 92ms/step - accuracy: 0.9891 - loss: 0.0552 - val_accuracy: 0.8445 - val_loss: 0.6233 - learning_rate: 8.0000e-07

Right out of the bat, B0 model had a slightly harder time reducing loss and accuracy per epoch (as it completed 13 epochs) compared to B4 (with 8 epochs). We know that B4 takes longer to train per epoch compared to B0, but how does it compare to the differenting amount of epochs to complete training?

B4 took 4,275 seconds in total from all epochs, whilst B0 took 3,599 seconds. Still faster than B4, which makes sense as B4 is a more complex and layered model than B0. So in comparison, it takes about an extra +676 seconds to train B4 over B0, or more crucially, it's about 18.8% increase in training time if extrapolated from the 3,599 seconds of training time.

Again, we can see B4 slightly outperforming B0 model.

Evaluation comparison 2

- B4
790/790 ━━━━━━━━━━━━━━━━━━━━ 83s 104ms/step - accuracy: 0.8473 - loss: 0.6386
[0.6472657918930054, 0.8453861474990845]

- B0
790/790 ━━━━━━━━━━━━━━━━━━━━ 64s 81ms/step - accuracy: 0.8391 - loss: 0.6468
[0.653698742389679, 0.8375841379165649]

B4 again does outperform B0 by the tiniest margins with our full dataset evaluation. B4 val_accuracy is 0.8473, while B0 val_accuracy is 0.8391 - About a 1% improvement in prediction. But calculating the time taken percentage wise compared to B0 (64 seconds), it's a 29% increase in evaluation time for B4 alone, versus the 18.8% increase during fine tuning.

I also decided to calculate the feature extraction's time increase %, and it's shorter at 14.8%. This makes sense, as the model does not need to change a lot of weights, and can keep them the way they are. As the more weights and layers there are, the more time you need to spend changing them.

So is the 14.8% to 29% longer training, a pivotal thing to implement for such marginal improvement? Maybe not.

Loss and Accuracy comparison

![alt text]()

![alt text]()

NOTE: The left side of the graph is B4 model, and right is B0.

When comparing the graph, again val_accuracy hasn't made substantial improvements between the two models, however it's already hard enough to make more improvements in the first place, due to complexity and number of classes. What we can very clearly see is the 'overfitting' of our training_accuracy/loss. B4 model is clearly a lot better at remembering and predicting the images it has seen before compared to B0, though they're still quite similarly tied on unseen data performance, with B4 slightly outperforming B0.

4. Name one important benefit of mixed precision training, and how it takes place?

I've already answered this question multiple times, so I will copy paste my exerpt from my blog to save time -

There is a form of model training called mixed_precision, where instead of using float32 to train the model, we use a mix of float16 as well. Using float16 halves the number of bits to calculate, therefore saving tons of time calculating very specific values.

However, the output layer must stay at float32 because of how big the values can get. Having float16 will overflow and return inf instead of a proper value.

This can speed up training 2 to 4 times, however its only limited to if you’re rich enough to afford the powerful GPUs. If running on lower end GPUs and CPUs alone, it’ll take much longer than if done on float32, because the computer needs to calculate for the switch from float32 to float16 which takes a lot of time (but doesn’t take much effort for the good GPUs).